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ABSTRACT 


We  conducted  a  study  to  examine  the  effects  of  target  cueing  and  conformality  with  a 
hand-held  or  head-mounted  display  to  determine  their  effects  on  visual  search  tasks  requiring 
focused  and  divided  attention.  Eleven  military  subjects  were  asked  to  detect,  identify,  and  give 
azimuth  information  for  targets  hidden  in  terrain  presented  in  a  simulated  far  domain 
environment  while  performing  a  monitoring  task  in  the  near  domain  using  either  a  helmet- 
mounted  display  or  hand-held  display.  The  results  showed  that  the  presence  of  cueing  aided  the 
target  detection  task  for  expected  targets  but  drew  attention  away  from  the  presence  of 
unexpected  targets  in  the  environment  representing  a  form  of  cognitive  tunneling.  This  effect 
was  mediated  by  the  display  platform  used,  such  that  cognitive  tunneling  was  reduced  when 
subjects  were  using  the  hand-held  display.  Additionally,  the  results  showed  that  the  presence  of 
cueing  hindered  performance  on  the  secondary  task. 

1.  INTRODUCTION 

The  ground  soldier  of  the  future  will  be  assisted  by  electronic  information  delivered  to 
the  field  (National  Research  Council,  1997).  An  important  issue  concerns  the  display  platform 
upon  which  or  within  which,  this  information  will  be  presented,  in  a  format  that  it  can  be  most 
useful,  and  least  disruptive  of  other  tasks.  At  least  two  solutions  are  available.  One  is  to 
capitalize  upon  existing  helmet-mounted  display  (HMD)  technology  (e.g.,  for  providing  night 
vision  information),  in  order  to  add  to  the  HMD  additional  electronic  information,  regarding 
mission  requirements,  terrain,  or  the  evolving  aspects  of  the  battle.  The  other  is  to  provide 
similar  information  on  an  electronic  hand-held  display. 

The  contrast  between  these  two  display  platforms,  parallels  a  contrast  that  has  been 
examined  more  extensively  in  the  aircraft  and  to  a  lesser  extent  in  ground  vehicles,  between 
presenting  information  on  a  head  up  display  (HUD),  overlapping  the  forward  field  of  view, 
versus  the  head  down  instrument  panel  (Weintraub  &  Ensing,  1992;  Wickens,  1997). 
Experimental  evaluations  of  this  comparison  in  the  HUD  studies  has  revealed  a  tradeoff  between 
two  critical  attentional  variables:  the  costs  to  focused  attention,  related  to  the  clutter  of 
overlapping  imagery,  when  information  is  presented  head  up  so  that  it  is  superimposed  on  the 
outside  scene  creating  often  a  cluttered  view,  and  the  costs  to  divided  attention,  or  information 
access,  when  information  is  presented  head  down,  and  the  operator  (pilot  or  driver)  must  now 
scan  between  the  display  and  the  outside  world. 

Research  that  has  compared  these  two  platforms  (Wickens  &  Long,  1995;  Martin- 
Emerson  &  Wickens,  1997;  May-Ververs  &  Wickens,  1998;  Fadden  &  Wickens,  1997),  has 
revealed  that  the  costs  of  scanning,  associated  with  head  down  presentation,  generally  outweigh 
the  costs  of  clutter,  associated  with  head  up  presentation,  thereby  generally  favoring  the  latter. 
However  such  research  also  indicates  that  the  clutter  costs  become  greater  and  more  disruptive, 
as  more  information  is  added  to  the  HUD  (May-Ververs  &  Wickens,  1998),  and  that  these  costs 
are  also  more  strongly  realized  in  detecting  events  in  the  world  if  those  events  are  unexpected, 
and  not  salient  (Wickens  &  Long,  1995;  Wickens,  1997). 

Research  on  HUDs  has  also  revealed  that  the  clutter  costs  can  be  moderated,  or  even 
eliminated,  through  the  use  of  conformal  imagery.  That  is,  information  which  has  direct  spatially 
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defined  referents  in  the  world  beyond.  Such  imagery  has  sometimes  been  characterized  to  create 
an  augmented  reality  (Drasic  &  Milgram,  1996),  in  the  sense  that  the  far  domain  imagery 
(reality)  that  is  directly  viewed,  is  augmented  by  computer  imagery  that  indicates  or  highlights 
particular  locations,  objects,  or  dimensions  within  that  reality. 

Two  different  forms  of  such  augmented  reality  or  conformal  imagery  may  be  provided: 
that  which  conforms  to  relatively  enduring  characteristics  of  the  far  domain  environment,  such  as 
a  horizon  line,  a  compass  scale,  or  the  contours  of  the  terrain;  and  that  which  conforms  to  the 
location  (or  identity)  of  relatively  transient  entities  within  the  far  domain  -  for  example  a  desired 
ground  track  or,  in  the  experiment  we  report  here,  cueing  the  estimated  location  of  particular 
enemy  targets.  Of  course  such  cueing  can  readily  be  presented  head  down  as  well  as  head  up. 
However  in  the  head  down  position,  it  appears  less  "natural"  and  direct,  and  requires  a  greater 
degree  of  cognitive  transformation,  to  use  the  cue  to  identify  a  direct  position  in  space.  Indeed  in 
the  HUD  research,  we  have  found  that  the  benefits  of  conformal  cueing  (locating  the  runway)  are 
considerably  enhanced  when  presented  in  a  head  up,  rather  than  at  a  head  down  location 
(Wickens  &  Long,  1995;  Fadden  &  Wickens,  1997). 

The  effectiveness  of  conformal  and  non  conformal  cueing  in  an  HMD  was  examined  by 
Yeh,  Wickens,  and  Seagull  (1998).  In  their  experiment,  soldiers  sat  in  a  virtual  environment 
(CAVE),  and  were  asked  to  scan  for,  and  detect  a  series  of  targets  (soldiers,  tanks,  mines  and 
nuclear  devices)  that  could  appear,  partially  masked,  against  the  mountainous  terrain  surrounding 
them.  Concurrently  they  performed  a  secondary  task.  The  investigators  found  that  when  the 
imagery  was  conformal  or  "world  referenced",  there  was  an  overall  benefit  to  target  detection. 
There  was  also  an  overall  benefit  of  target  cueing  (independently  of  the  degree  of  conformality), 
such  that  a  reticle  pointing  to  the  location  of  targets  facilitated  their  detection.  However,  a 
somewhat  disconcerting  result  was  that  cueing  in  particular  imposed  a  COST  on  the  detection  of 
simultaneously  viewable  uncued  targets  of  higher  priority,  the  nuclear  devices.  Yeh,  Wickens, 
and  Seagull  attributed  this  cost  to  a  sort  of  attentional  tunneling,  resulting  from  the  very 
compelling  nature  of  the  cue,  directing  the  soldier's  attention  to  the  cued  location,  but  in  the 
process,  disrupting  the  need  to  maintain  a  broader  scan,  to  detect  the  infrequent,  but  HIGHLY 
IMPORTANT  target.  The  investigators  also  found  that  the  cost  of  cueing  was  slightly  modulated 
by  the  nature  of  the  display,  enhanced  in  the  screen-referenced,  relative  to  the  world-referenced 
display. 

A  similar  finding,  in  a  different  environment,  was  observed  by  Bossi,  Ward,  Parkes,  and 
Howarth  (1997).  In  a  vehicle  driving  simulator,  they  found  that  highlighting  the  center  of  the 
highway,  via  an  enhanced  vision  system,  facilitated  the  highlighted  task  (road  following),  but  at 
the  expense  of  detection  of  more  peripheral  targets. 

Yeh,  Wickens,  and  Seagull  did  not  look  at  the  hand-held  display  (i.e.,  one  without 
overlapping  imagery)  in  their  experiment,  and  thus  their  findings  cannot  as  address  the  issue  of 
the  clutter-scan  (focused-divided  attention)  tradeoff  that  underlies  much  of  the  HUD  research. 
Nor  can  their  results  directly  address  how  such  a  tradeoff  may  be  moderated  by  features  of  the 
conformal  (world  referenced)  imagery,  in  a  way  that  might  guide  designers  to  present 
information  that  is  most  usable,  and  least  disruptive  of  the  soldier's  need  to  monitor  the 
environment  beyond. 
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This  then  is  the  specific  objective  of  the  current  experiment,  in  which  participants 
perform  the  same  combination  of  a  target  detection  and  identification  task  and  a  secondary 
monitoring  task,  that  was  used  by  Yeh,  Wickens,  and  Seagull.  In  one  condition  they  do  so  using 
a  helmet  mounted  display  with  conformal  imagery  (i.e.,  augmented  reality);  in  a  second 
condition,  the  same  information  is  presented  on  a  hand-held  electronic  map.  Either  display 
location  is  sometimes  augmented  by  target  cueing,  and  both  cued  and  uncued  targets  may 
sometimes  be  accompanied  by  a  higher  priority,  but  unexpected  (i.e.,  low  frequency)  nuclear 
weapon.  Our  interests  are  in  the  extent  to  which  (a)  clutter  and  scanning  trade-off  with  each 
other,  (b)  the  expected  benefits  of  target  cueing  are  enhanced  when  the  cues  are  rendered  in  the 
HMD,  rather  than  on  the  hand-held  display,  and  (c)  the  extent  to  which  benefits  of  cueing  may 
be  offset  by  costs  of  detecting  uncued,  but  high  priority  targets,  and  that  these  costs  themselves 
may  be  modulated  by  the  platform  of  the  display. 

2.  METHOD 


2.1.  Subjects 

Eleven  Army  personnel  (three  officers  and  eight  Reserve  Officers  Training  Corps  cadets) 
at  the  University  of  participated  in  the  experiment. 

2.2.  Task  Overview 


The  task  performed  by  subjects  consisted  of  three  stages:  (a)  target  detection,  (b)  target 
identification,  and  (c)  target  heading.  The  target  detection  task  (the  primary  task)  required 
subjects  to  scan  the  display  looking  for  any  one  of  four  target  objects:  three  of  the  targets  were 
presented  on  a  total  of  90%  of  the  trials  (30%  each)  and  were  therefore  expected;  the  fourth  was 
presented  only  10%  of  the  time  and  was  unexpected.  Subjects  were  not  told  which  target  to 
search  for.  While  searching  for  the  target,  subjects  were  asked  to  perform  a  secondary 
monitoring  task,  displayed  on  the  simulated  HMD,  which  stopped  once  the  target  was  found. 

Cueing  of  the  target’s  location  was  presented  for  half  the  expected  targets  to  aid  the 
detection  task;  the  unexpected  target  was  never  cued.  Targets  were  presented  serially;  only  one 
target  was  displayed  at  any  time,  except  in  the  case  of  the  unexpected  target,  which  was  always 
presented  in  conjunction  with  an  expected  target.  However,  only  one  object  was  detected  per 
trial.  Subjects  were  instructed  that  detecting  the  unexpected  target  -  a  nuclear  weapon  -  took 
precedence  over  standard  target  detection. 

Once  the  target  was  found,  the  subject  was  required  to  identify  the  target  as  either  friend 
or  foe  and  give  the  target’s  heading,  the  current  compass  direction  of  the  target  with  respect  to 
his  current  location. 
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2.3.  Apparatus 


The  terrain  was  displayed  on  the  walls  of  the  Cave  Automatic  Virtual  Environment 
(CAVE),  a  10xl0x9-foot  room  sized  video  environment.  The  subject  was  seated  in  the  center  of 
the  CAVE.  A  DataVisor  VGA  HMD  was  used  for  the  HMD  condition.  Although  symbology  was 
displayed  monocularly  to  the  subjects’  right  eye,  subjects’  field  of  view  (i.e.,  the  amount  of 
information  available  to  both  eyes)  was  constrained  to  30°.  The  symbology  was  presented  in  a 
25°  field  of  view. 

In  the  hand-held  condition,  subjects  viewed  the  image  presented  on  a  hand-held  display 
with  a  2.5”  screen.  They  were  required  to  wear  head-tracked  shutter  glasses,  which  were  not 
turned  on,  but  reduced  the  saliency  of  the  far  domain  targets  to  equal  that  of  the  view  through  the 
HMD.  The  symbology  on  the  hand-held  display  was  visible  to  both  eyes. 

2.4.  Displays/Tasks 

The  displays  were  created  from  static  two-dimensional  renderings  of  three-dimensional 
images  depicting  hilly  terrain.  The  terrain  was  developed  using  geographical  data  of  Austin,  TX, 
Detroit,  MI,  and  Jordan  Valley,  UT,  downloaded  from  the  U.S.  Geological  Survey  web  site.  The 
target  stimuli,  shown  in  Figure  6.4.1,  were  placed  in  the  terrain. 


(a)  Tank:  Friend  Foe  (b)  Soldier:  Friend  Foe 

i. 

®  1 

(c)  Fand  Mine  (d)  Nuclear  Device 


Figure  2.4.1.  Stimuli:  (a)  Tanks  (cued),  (b)  Soldiers  (uncued),  (c)  Fand  Mine,  (d)  Nuclear 
Device. 


The  tanks,  soldiers,  and  nuclear  devices  were  camouflaged,  i.e.,  colored  in  shades  of 
brown,  green,  and  black;  land  mines  were  presented  in  black.  Since  the  shading  of  the  terrain 
varied,  the  intensity  of  the  targets  was  adjusted  adaptively  at  each  location  so  that  the  contrast 
ratios  between  the  target  and  the  terrain  were  similar  for  all  targets.  The  greater  salience  of  the 
nuclear  device  was  insured  by  presenting  them  at  a  higher  contrast  ratio  with  the  background 
than  the  other  three  targets.  The  location  of  tanks  and  50%  of  the  land  mines  were  cued  with  an 
arrow  pointing  in  the  direction  of  the  target  based  on  the  subject’s  current  head  position.  All 
soldiers  and  50%  of  the  land  mines  were  uncued.  The  presence  of  cueing  was  randomized  (i.e., 
unpredictable)  over  trials. 
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Friend-or-foe  identification  was  based  on  the  direction  in  which  the  target  was  pointing. 
Friendly  targets  pointed  towards  the  left  and  enemy  targets  pointed  towards  the  right.  No 
identification  was  required  if  the  target  was  a  land  mine  or  nuclear  device. 

Head  Up  Display 

An  example  of  the  head-up  (HMD)  display  is  presented  in  Figure  2.4.2,  which  depicts  the 
field  of  view  of  the  symbology,  the  horizon  line,  the  cueing  arrow,  and  the  box  containing  the 
secondary  task. 


Figure  2.4.2.  HMD  display. 


In  Figure  2.4.2,  the  pictures  show  symbology  presented  in  the  HMD  superimposed  onto 
the  terrain  presented  on  a  CAVE  wall.  Symbology  was  presented  in  green.  The  visual  region  of 
HMD-depicted  information  was  only  25°  laterally  x  30°  vertically.  The  field  of  view  presented  to 
the  subjects  was  slightly  less  than  the  40°  available  in  night  vision  devices  and  significantly 
smaller  than  the  60°  required  by  the  Army  for  the  Land  Warrior  System. 

Heading  was  presented  conformally  (i.e.,  world-referenced)  with  respect  to  the  horizon 
line.  The  four  cardinal  directions  were  marked  on  each  heading  tape.  Note  that  the  heading 
information  in  Figure  2.4.2  was  superimposed  on  the  true  horizon  line,  and  as  a  result,  the 
location  of  the  heading  tape  on  the  HMD  changed  as  the  subject  moved  his  head  vertically  in 
order  to  examine  the  environment. 
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On  cued  trials,  a  cue  was  presented  to  signal  the  current  lateral  and  vertical  location  of  a 
target  with  respect  to  the  subject’s  head  orientation.  For  example,  if  the  target  was  presented  to 
the  right  of  the  subject,  then  a  right  pointing  arrow  appeared  on  the  HMD  as  shown  in  Figure 
2.4.2,  indicating  the  presence  and  general  direction  of  a  target.  If  the  target  was  above  and  to  the 
right  of  the  subject  -  e.g.,  located  on  top  of  a  mountain,  then  an  arrow  pointing  towards  the  upper 
right  corner  of  the  HMD  appeared.  The  arrow  would  change  its  orientation  accordingly  as  the 
subjects  moved  his  head  toward  the  target.  Note  that  targets  appearing  directly  in  front  of  the 
subject  within  the  forward  field  of  view  were  not  cued  by  this  symbology  but  rather  designated 
by  a  target  lock-on  indicator,  or  reticle,  which  will  be  discussed  later. 

Cueing  information  was  presented  in  a  partially  head-referenced  format.  A  cueing  arrow 
positioned  on  the  perimeter  of  the  screen  display,  pointed  directly  toward  the  3D  location  of  a 
target.  In  this  example,  the  arrow  would  indicate  that  the  target  was  to  the  right  and  above.  The 
cueing  arrow  could  be  positioned  at  the  edges  of  the  perimeter  of  a  circle  whose  diameter 
subtends  21°  of  visual  angle. 

Once  the  target  was  in  the  subject’s  field  of  view,  i.e.,  visible  through  the  HMD,  a  target 
lock-on  reticle  appeared  on  the  display  conformally  as  shown  in  Figure  2.4.3. 


Figure  2.4.3.  Target  lock-on.  Note  that  the  tank  is  in  the  far  domain  and  is  being  viewed  through 
the  HMD. 
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As  Figure  2.4.3  shows,  the  lock-on  reticle  was  displayed  over  the  actual  object.  Due  to 
variability  in  the  head  tracker  attached  to  the  HMD,  the  cueing  was  somewhat  imprecise;  thus, 
the  lock-on  reticle  could  be  slightly  off  target  at  approximately  0°-3°  in  the  x-,  y-,  or  z-directions. 
Our  previous  research  (Yeh,  Wickens,  &  Seagull,  1998)  had  revealed  that  this  imprecision 
created  no  difficulties  in  target  detection  since,  in  any  case,  the  target  was  within  foveal  vision 
when  the  lock-on  reticle  was  fixated.  The  lock  on  reticle  was  not  used  to  signal  the  presence  of 
any  uncued  targets  which  might  appear  in  the  subject’s  forward  field  of  view. 

Hand-held  display 

An  example  of  the  hand-held  display  is  presented  in  Figure  2.4.4. 


Figure  2.4.4.  Hand-held  display. 


As  Figure  2.4.4  shows,  the  hand-held  display  provided  subjects  with  a  simple  diagram  of 
the  world,  heading  information,  cueing  information,  and  the  secondary  task.  The  information  on 
the  hand-held  was  presented  non-conformally,  i.e.,  if  the  subject  would  move  the  display  in  the 
environment,  the  positioning  of  the  symbology  (heading  and  cueing  data)  did  not  change. 
Representation  of  the  walls  of  the  CAVE  (the  lines  at  the  edge  of  the  display)  were  drawn  in 
blue,  and  the  heading  information  was  presented  in  white  against  a  black  background. 
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On  cued  trials,  a  cue  was  presented  to  signal  the  current  lateral  location  of  a  target.  This 
cue  was  independent  of  the  subject’s  head  or  hand  orientation;  in  other  words,  the  presentation  of 
the  cue  did  not  change  when  the  target  was  within  the  subject’s  field  of  view.  Thus,  the  arrow, 
which  in  the  head-up  condition  cued  subjects  to  the  target’s  location,  was  not  included  in  this 
display. 

2.5.  Secondary  Task 


Subjects  were  given  a  secondary  monitoring  task,  to  perform  continuously  throughout  the 
experiment.  In  the  HMD  condition,  the  task  was  presented  monocularly  to  the  right  eye;  in  the 
hand-held  condition,  the  task  was  presented  to  both  eyes.  Note  that  the  task  was  a  two- 
dimensional  task;  thus,  viewing  the  secondary  task  with  two  eyes  versus  one  would  not  result  in 
different  perceptions  of  the  same  image. 

Subjects  were  told  that  enemy  troops  were  tracking  their  location  by  using  radio 
frequency  as  input.  Thus,  as  the  subjects  searched  for  targets,  they  were  also  required  to  jam  the 
enemy’s  radar  frequency  so  that  they  remained  undetected.  To  do  this,  subjects  needed  to 
monitor  a  horizontal  bar,  presented  at  the  lower  left  edge  of  the  HMD  (as  shown  in  Figure  2.4.1) 
and  in  the  center  of  the  hand-held  display  (as  shown  in  Figure  2.4.4.).  The  solid  bar  gradually 
grew  longer  horizontally,  filling  in  the  rectangle  from  left  to  right.  When  it  passed  the  first 
marker,  subjects  had  5  seconds  to  jam  the  enemy’s  frequency  by  responding  with  a  button  press. 
Responding  before  the  solid  bar  passed  the  first  marker  had  no  effect.  The  solid  bar  increased  at 
a  variable  rate  created  as  the  sum  of  four  sine  functions.  The  bar  reached  the  first  marker 
between  three  and  five  seconds  from  the  start  of  the  secondary  task.  Once  subjects  responded  to 
the  task,  the  bar  would  reset.  The  task  continued  until  the  target  was  detected. 

2.6.  Experiment  Design 

The  experiment  was  a  mixed  design  as  shown  in  Table  2.6.1. 


Table  2.6.1.  Experimental  design. 


Within  Subjects 

Display 


Head-Up 

Head-Down 

Reward 

Between 

TARGET TYPE 

CUED  UNCI 

T  ank  M  ine  M  ine  Soldier 

^ -  Expected  - y 

JED 

Nuclear 

Device 

Unexpected 

TARGET  TYPE 

Subjects 

No  Reward 

TARGET  TYPE 

TARGET  TYPE 
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As  Table  2.6.1  shows,  the  presentation  of  display  [head-up  (HMD)  versus  head-down 
(hand-held)]  and  the  manipulation  of  target  type  (cued  versus  uncued  targets,  high  versus  low 
expectancy)  were  examined  within  subjects.  The  manipulation  of  reward  was  analyzed  between 
subjects.  The  first  three  subjects  in  the  experiment  reported  that  the  information  presented  on  the 
hand-held  did  not  add  much  to  their  task,  and  were  thus  scanning  the  environment  naturally;  as  a 
consequence,  the  rate  of  misses  for  the  secondary  task  was  at  26%  for  the  hand-held  display. 
Thus,  we  added  an  incentive;  subjects  were  ranked  based  on  their  performance  on  the  secondary 
task  and  could  earn  $40,  $20,  or  $10  for  first,  second,  or  third  place  in  terms  of  task  performance. 
To  prevent  subjects  from  responding  to  the  secondary  task  randomly  (e.g.,  selecting  the  button 
every  3  seconds  or  so),  the  remaining  subjects  were  told  that  they  would  be  penalized  if  they 
responded  either  too  early  or  too  late.  The  secondary  task  was  present  on  all  trials. 

Six  different  terrain  views,  created  from  taking  static  “pictures”  at  different  locations  of 
topographical  regions,  were  used  in  the  experiment.  For  each  viewing  condition,  subjects  were 
presented  with  one  practice  block,  consisting  of  ten  search  trials,  and  ten  experimental  blocks, 
each  containing  a  set  of  twenty  search  trials.  The  presentation  of  target  stimuli  (i.e.,  tanks,  land 
mines,  and  soldiers)  was  serial  -  that  is,  only  one  target  was  presented  per  trial  and  subjects 
searched  the  three  walls  of  the  CAVE  until  it  was  located.  The  exception  was  the  presentation  of 
the  nuclear  device,  which  would  appear  concurrently  with  one  of  the  other  targets. 

In  the  practice  block,  subjects  viewed  ten  targets,  presented  serially.  The  targets  consisted 
of  three  tanks,  three  soldiers,  and  three  land  mines,  each,  and  one  nuclear  device.  Tanks,  soldiers, 
and  land  mines  appeared  once  on  each  of  the  three  walls.  Each  experimental  block  consisted  of  a 
total  of  20  targets;  6  each  of  tanks,  soldiers,  and  land  mines  and  2  nuclear  devices.  Half  the  tanks 
and  half  the  soldiers  were  friendly  -  the  other  half  were  enemy.  On  a  random  half  the  trials, 
cueing  was  present.  This  was  the  case  for  all  tanks  and  half  of  the  mines.  Thus,  the  presence  of 
the  cueing  symbol  provided  subjects  with  a  partial  reduction  of  uncertainty  of  target  type.  Each 
object  appeared  twice  on  each  wall,  except  for  the  nuclear  device  which  appeared  once  on  the 
left  wall  and  once  on  the  right  wall.  Targets  were  presented  serially,  except  for  the  nuclear 
device,  which  was  presented  in  conjunction  with  either  a  cued  target  (tank)  or  an  uncued  target 
(soldier).  As  it  was  an  “unexpected”  target,  the  nuclear  device  was  presented  within  15°  of  either 
the  tank  or  the  soldier  with  which  it  was  presented  to  maximize  the  likelihood  that  the 
unexpected  target  would  appear  in  the  subject’s  field  of  view  as  the  subject  searched  for  the 
target  (e.g.,  if  the  target  was  a  tank  located  in  the  center  of  the  left  wall,  the  nuclear  device  would 
have  been  positioned  to  the  right  of  the  tank  so  that  the  subject’s  field  of  view  would  pass  over 
the  unexpected  target  as  he  moved  his  head  from  the  center  wall  to  the  left  wall).  Subjects  were 
told  that  a  nuclear  device  could  be  present  on  any  of  the  trials. 

2.7  Procedure 


The  experiment  took  approximately  2.5  hours  during  which  subjects  were  given  the 
instructions  for  the  experiment  and  then  performed  the  experiment.  Subjects  were  instructed  to 
pretend  that  they  were  scouts,  sent  to  search  for  enemies  and  allies  in  unfamiliar  territory.  Their 
primary  task  was  to  find  the  targets,  identify  them  as  friend  or  foe,  if  relevant,  and  send 
information  back  to  their  troop  regarding  the  objects’  position.  Their  secondary  task,  was  to 
monitor  a  radio  frequency  display,  which  provided  data  as  to  how  close  the  enemy  was  in 
tracking  their  position. 


9 


Subjects  interacted  with  the  display  using  a  wand  and  shutter  glasses.  A  diagram  of  the 
wand  is  presented  in  Figure  2.7.1. 


Figure  2.7.1.  The  wand. 


The  wand  has  three  buttons  and  a  pressure-sensitive  joystick.  Only  the  buttons  were  used 
during  the  experiment  to  make  responses.  The  joystick  was  not  used  at  all. 

While  searching  for  the  target,  subjects  responded  to  the  secondary  task  by  pressing  the 
right  wand  button.  To  indicate  that  a  target  was  detected,  subjects  pressed  the  left  button  on  the 
wand.  For,  t  target  identification  task,  subjects  pressed  the  left  button  on  the  wand  again  if  the 
target  was  foe,  the  center  button  if  the  target  was  friendly,  or  the  right  button  in  the  case  of  a 
nuclear  device.  Subjects  did  not  need  to  identify  whether  the  target  was  a  tank,  soldier,  or  land 
mine.  Note  that  the  button  pressed  for  friend  and  foe  identifications  corresponded  to  the  direction 
the  object  was  pointing,  e.g.,  subjects  pressed  the  left  button  if  the  tank  or  soldier  was  pointing 
left.  Once  the  target  was  detected  (land  mine)  or  identified  (tank,  soldier,  or  nuclear  device), 
subjects  verbally  reported  its  location  by  stating  the  target’s  bearing. 

Once  the  target  was  detected  and  reported,  the  display  was  darkened.  When  the  subject’s 
head  was  centered,  a  subsequent  trial,  containing  a  new  target,  was  initiated. 

After  each  twenty  trial  block,  subjects  were  asked  to  “describe”  the  location  of  the  targets 
they  had  encountered  within  the  environment  to  their  commanding  officer  by  selecting  one  of 
four  pictures  of  the  environment,  one  of  which  depicted  the  objects  in  the  same  location  as  in  the 
environment  they  had  seen.  Of  the  three  incorrect  pictures,  one  showed  the  tanks  placed  in 
different  positions,  another  presented  the  soldiers  in  different  locations,  and  the  third  depicted  the 
land  mines  in  incorrect  sites.  Not  all  the  targets  were  presented.  That  is,  targets  presented  on 
nuclear  device  trials  were  omitted  from  the  pictures  since  it  was  not  known  which  target  subjects 
would  detect  in  the  nuclear  device  trials  -  i.e.,  would  subjects  see  the  missile  or  would  the  tank 
or  soldier  appearing  with  the  missile  capture  their  attention  instead. 

The  order  in  which  subjects  viewed  the  symbology  (head-up  versus  head-down)  was 
counterbalanced. 

2.8.  Performance  Measures 


The  dependent  variables  collected  from  the  primary  target  search  task  were  response  time 
and  accuracy  for  target  detection,  target  identification,  and  target  heading.  In  order  to  determine 
whether  the  symbology  influenced  the  amount  of  scanning  in  the  environment,  data  describing 
the  amount  of  head  movement  along  the  x-,  y-,  and  z-  axes  were  collected.  Additionally,  data 
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concerning  the  number  of  times  and  the  amount  of  time  the  target  was  in  the  view  (within  60°, 
21°,  and  15°)  were  collected.  Note  that  the  center  points  for  the  aforementioned  view  angles  are 
at  the  center  of  the  HMD  and  shutter  glasses,  rather  than  the  center  of  the  eyes.  Thus,  it  was 
possible  for  a  target  to  pass  through  the  area  in  the  center  15°  of  the  shutter  glasses  and  go 
unnoticed  by  the  subject,  if  his  eyes  were  rotated  away  from  the  forward  axis  of  the  head. 

The  measures  collected  from  the  secondary  task  were  response  time  and  accuracy.  Since 
each  subject  took  a  different  amount  of  time  in  detecting  the  targets,  the  number  of  frequency 
jamming  events  varied.  Thus,  accuracy  for  the  task  was  calculated  as  a  proportion  of  the  number 
of  hits  to  the  number  of  total  frequency  jamming  tasks  viewed. 

Finally,  measures  for  the  global  positioning  task  were  response  time,  accuracy,  and 
subjects’  confidence  ratings  of  their  responses. 

3.  RESULTS 

The  data  were  examined  in  order  to  determine  the  effects  of  expectancy  and  cueing  on 
target  detection  and  how  well  attention  could  be  allocated  between  the  near  and  far  domains. 
Differences  in  display  (HMD  vs.  hand-held)  were  hypothesized  to  mediate  these  effects.  Since  it 
was  possible  for  subjects  to  mistake  a  terrain  feature  for  an  object,  trials  with  heading  errors  of 
azimuth  estimation  greater  than  ±20°  were  assumed  to  result  from  this  confusion,  were  scored  as 
incorrect  and  replaced  with  the  subject’s  mean  response  time  for  like  targets  in  that  particular 
block  (i.e.,  involving  the  same  terrain)  displayed  on  the  same  wall.  This  was  approximately  5% 
of  the  trials.  Additionally,  outliers  which  were  greater  than  ±3  standard  deviations  from  the  mean 
were  replaced  in  the  same  way;  this  was  approximately  1%  of  the  trials. 

The  total  data  set  represented  ten  dependent  variables,  consisting  of  response  time  and 
accuracy  measures  for  the  primary  tasks  of  target  detection,  identification,  and  location,  the 
frequency  jamming  secondary  task,  and  the  global  positioning  recognition  task.  These  dependent 
variables  were  influenced  by  multiple  factors  (independent  variables): 

•  Target  type,  which  could  be  subdivided  into  comparisons  of  expected  vs.  unexpected 
targets  and  cued  vs.  uncued  targets 

•  Display:  HMD  vs.  hand-held 

•  Wall:  left,  center,  or  right 

Because  we  do  not  hypothesize  that  all  dependent  variables  would  plausibly  be 
influenced  by  all  independent  variables  (or  if  they  were,  such  influences  would  not  be  of 
theoretical  or  practical  interest),  we  do  not  report  full  ANOVAs  on  all  dependent  variables. 
Instead,  we  parse  the  presentation  of  results  into  six  categories: 

1.  Effects  of  target  type  (expectancy  and  cueing)  and  display  on  the  primary  and 
secondary  tasks  (3. 1-3.5) 

2.  Effects  of  wall  (3.6) 
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Within  each  of  these  sections,  we  present  and  describe  only  those  effects  (and  their 
interactions)  that  are  most  relevant  to  understanding  the  influence  of  display  augmentations  on 
target  detection. 

3.1.  Expectancy 

The  effects  of  a  subject’s  expectation  of  a  target  was  examined  by  comparing  detection 
performance  for  tanks  and  soldiers  -  both  highly  expected  targets  -  with  detection  of  nuclear 
devices  -  infrequent,  low  expectation  targets.  Subjects  were  instructed  that  the  latter  were  of 
higher  priority.  The  nuclear  device  trials  were  separated  into  two  classes  based  on  whether  the 
nuclear  device  was  presented  concurrently  with  a  tank  or  with  a  soldier.  Although  mines  were 
also  expected  targets,  the  mine  trial  data  were  not  used  for  this  analysis  since  mines  were  cued  on 
half  the  trials,  thus  confounding  the  measure  of  expectation.  Note  also  that  the  presentation  of  the 
unexpected  targets  (nuclear  devices)  never  occurred  concurrently  with  a  mine.  No  comparisons 
were  made  between  the  two  expected  targets  (tanks  and  soldiers)  as  variables  affecting 
performance  could  not  be  attributed  solely  to  cueing,  i.e.,  tanks  were  cued  and  soldiers  were  not, 
but  there  were  also  possible  confounding  differences  attributable  to  in  the  physical  appearance  of 
the  stimuli.  The  direct  effects  of  cueing  will  be  examined  in  the  analysis  of  mine  detection 
(Section  3.2). 

A  2  (display:  HMD  vs.  hand-held)  x  4  (target  type:  expected  and  cued  (tank),  expected 
and  uncued  (soldier),  unexpected  with  cued  (nuclear  device  presented  with  a  tank),  unexpected 
with  uncued  (nuclear  device  presented  with  a  soldier))  within  subjects  ANOVA  was  conducted 
on  the  accuracy  and  response  times  for  the  target  detection  task.  Figure  3.1.1  presents  the  effects 
of  display  and  target  type  on  response  time  (left)  and  accuracy  (right).  The  bars  in  the  figures 
show  ±1  standard  errors  from  the  mean. 
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Figure  3.1.1.  Response  time  and  accuracy  for  expected  and  unexpected  targets. 
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In  the  graph,  the  filled  symbols  are  responses  when  the  uncued  target  (soldier)  was 
present,  the  open  symbols  represent  responses  when  the  cued  target  (tank)  was  present,  and  the 
triangles  represent  the  data  points  for  the  nuclear  devices.  The  main  effect  of  display,  F(l,18)  = 
10.21,  p  =  .005,  suggested  an  advantage  for  the  hand-held  display,  and  the  main  effect  of  target 
type,  F(3,54)  =  32.67,  p  =  .0001,  suggested  that  the  most  rapid  responses  occurred  on  cued  trials. 
The  significant  interaction  between  display  and  target  type,  F(2,54)  =  16.26,  p  =  .0001  suggests 
that  the  cueing  effect  was  enhanced  when  using  the  HMD,  and  that  the  HMD  cost  was  only 
observed  on  cued  (i.e.,  tank)  trials.  More  detailed  comparisons  within  the  target  types  revealed 
no  difference  between  the  uncued  nuclear  weapons  and  the  targets  with  which  they  were  paired 
[nuclear  device  vs.  tank,  F(l,18)  =  .01,  p  =  .93;  nuclear  device  vs.  soldier,  F(l,18)  =  .76,  p  =  .40], 
although  a  slight  benefit  was  present  for  detecting  the  nuclear  weapons  over  the  tanks  and 
soldiers  when  subjects  used  the  hand  held  display,  F(l,18)  =  18.68,  p  =  .0004,  and  F(l,18)  = 
24.09,  p=  .0001. 

As  shown  in  Figure  3.1.1(b),  the  latency  advantage  for  the  nuclear  weapons  detection  was 
purchased  at  a  cost  for  accuracy.  The  main  effect  of  target  type,  F(3,54)  =  13.33,  p  =  .0001, 
reflected  this  cost  (a  25%  reduction  in  hit  rate  for  the  nuclear  weapons  compared  to  near  perfect 
detection  for  the  more  expected  soldiers  and  tanks).  More  specifically,  the  target  x  display 
interaction,  F(3,54)  =  7.32,  p  =  .003,  suggested  that  this  cost  was  amplified  only  when  the 
nuclear  device  was  paired  with  a  cued  tank  when  using  the  HMD. 

3.2  Benefits  of  Cueing 

In  order  to  determine  the  effect  of  cueing,  unconfounded  by  stimulus  type,  a  comparison 
of  the  detection  of  cued  versus  uncued  land  mines  was  conducted.  The  data  were  analyzed  using 
a  2  (display)  x  2  (cueing:  cued  vs.  uncued)  x  3  (wall:  left,  center,  and  right)  within  subjects 
ANOVA.  Figure  3.2.1  shows  the  results  for  the  target  detection  task. 
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Data  regarding  mine  detection  showed  a  large  response  time  benefit  for  target  cueing, 
F(l,18)  =  42.05,  p  =  .0001,  replicating  the  cueing  response  time  advantage  for  tanks  seen  in 
Figure  3.1.1.  There  was  a  marginal  effect  of  display,  F(l,  1 8)  =  3.75,  p  =  .07,  and  a  significant 
interaction  between  target  cueing  and  display,  F(l,18)  =  28.27,  p  =  .0001,  indicating  that  the 
cueing  benefit  was  enhanced  in  the  HMD,  relative  to  the  hand-held  display,  and  as  shown  in 
Figure  3.1.1,  the  HMD  costs  were  observed  for  uncued,  but  not  cued  targets. 

Analysis  conducted  on  the  accuracy  data  also  revealed  a  benefit  for  cueing,  F(l,  1 8)  = 
13.12,  p  =  .002.  There  were  no  differences  due  to  display,  F(l,18)  =  2.17,  p  =  .15,  nor  was  the 
interaction  between  cueing  and  display  significant,  F(l,  1 8)  =  .18,  p  =  .68. 

3.3  Divided  Attention:  Results  of  Secondary  Task  Performance 


In  order  to  determine  how  well  subjects  were  able  to  divide  their  attention  between 
information  presented  in  the  display  and  information  in  the  far  domain,  ANOVAs  were 
conducted  on  the  response  time  and  accuracy  data  for  the  secondary  task.  A  2  (display)  x  2 
(cueing)  x  2  (reward)  ANOVA  was  conducted  on  the  data  for  secondary  task  performance.  The 
latency  and  accuracy  with  which  subjects  responded  to  the  secondary  task  are  presented  in 
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Figure  3.3.1.  Response  time  and  accuracy  for  the  secondary  task. 


The  response  time  data  showed  no  effect  of  display,  F(l,18)  =  .17,  p  =  .68,  cueing, 
F(l,18)  =  .21,  p  =  .65,  reward,  F(l,  1 8)  =  1.62,  p  =  .22,  nor  an  interaction  between  cueing  and 
display,  F(l, 18)  =  1.91,  p=. 18. 

The  accuracy  data  revealed  no  effect  for  display,  F(l,19)  =  1.42,  p  =  .25,  but  a  marginal 
effect  for  target  cueing,  suggesting  that  subjects  were  able  to  focus  their  attention  more 
effectively  on  the  secondary  task  when  there  was  no  target  cueing  on  the  display,  F(  1,19)  =  3.42, 
p  =  .08.  The  interaction  between  display  and  cueing  was  not  significant,  F(l,19)  =  .34,  p  =  .56. 

The  results  for  the  reward  manipulation  are  presented  in  Figure  3.3.2. 
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Figure  3.3.2.  Accuracy:  reward  versus  no  reward. 


The  results  showed  that  performance  on  the  secondary  task  marginally  improved  when 
subjects  were  given  an  incentive,  F(l,19)  =  3.58,  p  =  .08,  but  the  data  from  both  groups  showed 
the  same  trend,  as  reported  above  in  the  context  of  Figure  3.1.1. 

3.4  Display  Effects  for  Target  Identification  and  Heading  Tasks 

The  results  presented  so  far  have  described  the  data  analysis  of  the  effects  of  cueing  and 
expectancy  on  allocating  attention  between  the  near  and  far  domains  to  detect  secondary  task 
events  and  targets  respectively.  Subjects  were  also  asked  to  perform  two  far  domain  tasks  in 
addition  to  target  detection  -  target  identification  and  target  heading  estimation  -  in  order  to 
determine  whether  the  use  of  conformal  or  non-conformal  imagery  could  facilitate  performance 
once  the  target  had  been  detected.  These  were  essentially  single  task  responses,  since  the 
secondary  task  was  inactive  during  this  phase.  Response  time  for  the  target  identification  task 
was  measured  by  the  time  delay  between  detection  and  identification.  As  noted,  identification 
was  not  required  for  the  mines  or  nuclear  devices,  since  these  were  always  assumed  to  be  hostile. 
A  2  (display)  x  2  (target  type:  tank,  soldier)  within  subjects  ANOVA  was  conducted  for  the 
target  identification  task.  Figure  3.4.1  shows  the  results  for  the  identification  task  for  the  tanks 
and  soldiers. 

As  Figure  3.4.1  shows,  there  was  no  effect  of  display  for  either  response  time  [F(  1,18)  = 
.45,  p  =  .50]  or  accuracy  [F(l,18)  =  .01,  p  =  .94].  However,  a  marginally  significant  interaction 
between  target  type  and  display  [F(l,l 8)  =  4.20,  p  =  .06]  was  present  for  the  accuracy  data,  such 
that  accuracy  in  identifying  the  tank  was  less  accurate  when  information  was  presented  on  the 
HMD  rather  than  the  hand-held  display,  but  that  the  accuracy  in  identifying  the  soldier  was 
influenced  little  by  display.  We  may  attribute  the  accuracy  cost  to  the  tank  to  the  cueing  reticle 
which  sometimes  cluttered  the  view  of  the  tank  (but  was  never  present  for  the  uncued  soldier). 
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Figure  3.4.1.  Response  time  and  accuracy  for  the  identification  task. 


Data  for  the  target  heading  task  were  analyzed  using  a  2  (display)  x  4  (target  type:  tank, 
soldier,  mine,  and  nuclear  device)  within  subjects  ANOVA.  When  determining  the  accuracy  for 
the  target  heading  task,  errors  in  heading  greater  than  ±10°  were  considered  incorrect.  The  results 
are  presented  in  Figure  3.4.2. 
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Figure  3.4.2.  Response  time  and  accuracy  for  the  target  heading  task. 


Heading  information  was  given  faster  by  subjects  using  the  HMD  rather  than  the  hand¬ 
held  display,  F(l,  16)  =  12.09,  p  =  .003.  A  main  effect  of  target  type  was  present  suggesting 
slower  responses  for  the  mines,  F(3,48)  =  8.50,  p  =  .001.  However,  an  interaction  between  target 
type  and  display  [F(3,48)  =  3.24,  p  =  .05]  suggested  that  this  slowing  was  only  observed  with  the 


HMD. 


The  influence  of  display  on  target  heading  accuracy  is  also  depicted  in  Figure  3.4.2.  The 
analysis  showed  a  main  effect  of  display,  F(l,16)  =  33.53,  p  =  .0001.  A  significant  interaction 
between  target  type  and  display  was  present,  F(3,48)  =  4.29,  p  =  .05,  such  that  heading  accuracy 
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for  the  three  expected  targets  (tanks,  soldiers,  and  land  mines)  was  higher  with  the  HMD  than 
with  the  hand-held  display,  but  the  opposite  was  true  for  the  unexpected  target  (nuclear  device). 

3.5  Global  Positioning  Task 

A  2  (display)  x  5  (terrain)  ANOVA  was  conducted  on  the  accuracy  for  the  global 
positioning  task,  as  presented  in  Figure  3.5.1. 
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Figure  3.5.1.  Accuracy  for  the  global  positioning  task. 


The  analysis  showed  an  effect  of  display,  F(l,98)  =  5.00,  p  =  .03,  such  that  recognition 
accuracy  was  greater  when  subjects  viewed  the  terrain  with  the  HMD  than  the  hand-held  display. 

Subjects  were  also  asked  to  give  a  confidence  rating  (1  =  not  confident,  5  =  confident)  as 
to  the  certainty  of  their  answer.  The  accuracy  data  were  then  converted  into  performance  scores 
based  on  subject’s  confidence  ratings  according  to  the  scale  presented  in  Table  3.5.1.  Scores 
were  decreased  with  lower  confidence  when  the  accuracy  was  correct,  and  decreased  with  higher 
confidence  when  accuracy  was  incorrect. 
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Table  3.5.1.  Confidence  score  based  on  response  accuracy. 


Correct 

Confidence 

Score 
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10 
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Incorrect 

Confidence 

Score 

1 
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4 

3 

3 

4 

2 

5 

1 
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Figure  3.5.2  shows  the  performance  ratings  for  the  global  positioning  task. 
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Figure  3.5.2.  Performance  ratings  for  the  global  positioning  task. 


The  data  showed  an  effect  of  display,  F(l,98)  =  4.6,  p  =  .03;  subjects  were  more  accurate 
and  more  confident  when  they  used  the  HMD  rather  than  the  hand-held. 

3.6  Scanning  Strategies 


The  data  for  detected  expected  targets  were  further  examined  in  order  to  provide  some 
insight  into  subject’s  scanning  strategies.  Analysis  was  conducted  in  order  to  determine  whether 
the  location  -  or  wall  -  on  which  the  target  item  was  presented  played  a  role  in  subjects’  ability 
to  detect  the  target.  A  2  (display)  x  4  (target  type:  tank,  soldier,  cued  mines,  and  uncued  mines)  x 
3  (wall)  repeated  measures  ANOVA  was  conducted  on  the  response  times  and  accuracy  for 
target  detection.  Figure  3.6.1  shows  the  results. 


Figure  3.6.1.  (a)  Response  time  and  (b)  accuracy  for  target  detection  due  to  wall. 
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The  results  revealed  no  effect  on  response  time  due  to  the  wall  on  which  the  target  was 
presented,  F(2,36)  =  1.90,  p  =  .18  but  a  significant  wall  x  target  type  interaction,  F(6,108)  =  5.95, 
p  =  .0001.  This  interaction  suggests  a  substantial  left  wall  cost  for  the  uncued  mine;  a  cost  not 
evident  for  the  other  three  expected  target  types.  Note  that,  as  shown  by  the  accuracy  data,  the 
uncued  mines  were  the  most  difficult  to  detect. 

The  results  for  the  accuracy  data  revealed  a  marginal  effect  due  to  wall,  F(2,36)  =  2.71,  p 
=  .08  such  that  target  were  detected  more  accurately  on  the  right  wall  (97%)  than  on  the  center 
wall  (93%),  F(l,18)  =  13.12,  p  =  .0001.  The  interaction  between  wall  and  target  type  was  not 
significant,  F(6,108)  =  1.78,  p  =  .15. 

Analysis  on  subjects’  head  motion  along  the  x-,  y-,  and  z-  axes  was  also  conducted  using 
a  2  (display)  x  4  (target  type:  tank,  soldier,  cued  mines,  and  uncued  mines)  repeated  measures 
ANOVA.  The  results  are  shown  in  Figure  3.6.2. 
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Figure  3.6.2.  Head  movement  along  the  x-,  y-,  and  z-  axes. 

The  data  revealed  that  wearing  an  HMD  constrained  head  movements;  subjects  moved 
their  head  to  scan  the  display  significantly  less  in  the  x-,  y-,  and  z-  directions  when  wearing  the 
HMD  than  when  using  the  hand-held  display  [x-axis:  F(  1,16)  =  21.50,  p  =  .0003;  y-axis:  F(l,16) 
=  17.20,  p  =  .0008;  z-axis:  F(l,  16)  =  35.14,  p  =  .0001], 
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4.  DISCUSSION 


The  current  experiment  was  conducted  to  determine  whether  manipulations  of  helmet 
mounted  and  hand-held  display  design  could  aid  tasks  of  focused  attention  in  the  near  and  far 
domains  as  well  as  divided  attention  between  the  two.  The  data  suggest  that  subjects’ 
expectancies  of  the  targets  and  the  presentation  of  cueing  information  aided  visual  search  for 
expected  targets  in  the  simulated  world,  but  that  cueing  sometimes  captured  attention  in  a  way 
that  resulted  in  a  cost  for  the  detection  of  unexpected  and  uncued  targets  in  the  far  domain.  The 
search  task  was  a  difficult  one  in  the  sense  that  subjects  were  searching  for  multiple  targets 
whose  identity  was  unknown.  Their  only  clue  available  to  reduce  uncertainty  was  that  if  a  cueing 
arrow  was  present  at  the  start  of  a  trial,  then  they  needed  to  search  for  a  tank,  land  mine,  or 
nuclear  device,  and  could  exclude  the  soldier  from  the  target  search.  If  the  cue  was  not  present, 
targets  in  the  far  domain  could  be  a  soldier,  land  mine,  or  nuclear  device.  The  greater  the 
potential  for  multiple  targets  in  the  scene,  the  more  mental  templates  the  subject  needed  to 
activate  in  order  to  complete  his  task,  and  hence,  the  harder  the  task. 

The  results  of  the  study  show,  not  surprisingly,  that  target  cueing  aided  the  detection  of 
cued  targets,  e.g.,  the  always-cued  tanks  were  better  detected  than  the  never-cued  soldiers  and 
the  cued  mines  were  better  detected  than  the  uncued  mines.  In  making  the  former  comparison,  it 
is  important  to  note  that  detection  of  soldiers  and  tanks  were  never  compared  in  the  uncued 
format,  so  that  similarities  or  differences  in  the  inherent  detectability  of  the  two  shapes  could  not 
be  determined,  and  were  therefore  confounded  with  cueing.  However,  a  cost  to  target  cueing  was 
present,  such  that  subjects  were  more  likely  to  overlook  a  high  priority  but  unexpected  target 
(nuclear  weapon),  when  it  was  paired  with  the  lower  priority  cued  target  (tank)  than  with  the 
lower  priority  uncued  soldiers,  as  reflected  in  the  accuracy  data  for  expected  versus  unexpected 
targets  shown  in  Figure  3.1.1.  This  effect  replicated  that  observed  by  Yeh,  Wickens,  and  Seagull 
(1998).  However,  Figure  3.1.1  also  reveals  that  this  cueing  cost  for  detection  of  unexpected 
targets  was  only  observed  with  the  HMD.  That  is,  this  effect  was  mediated  by  display,  such  that 
relative  to  the  HMD,  the  use  of  a  hand-held  display  improved  the  accuracy  of  detecting  a  rare  but 
important  uncued  target  in  the  presence  of  a  cued  target. 

We  can  try  to  interpret  the  HMD  cost  in  the  accuracy  for  detecting  unexpected  targets 
presented  concurrently  with  a  cued  target  in  terms  of  the  information  access  cost  imposed  by 
either  clutter  or  scanning.  One  hypothesis  is  that  the  imagery  on  the  HMD  increased  the  clutter  in 
the  forward  field  of  view  and  obscured  the  nuclear  device  even  if  it  did  not  necessarily  overlay  it. 
If  this  were  the  case,  then  we  would  expect  some  HMD  cost  to  nuclear  device  detection  to  be 
observed,  whether  cueing  was  present  or  not,  relative  to  performance  with  the  hand-held  display; 
but  this  cost  would  be  enhanced  when  cueing  was  present  (adding  to  the  clutter  on  the  display). 
The  accuracy  data  presented  in  Figure  3.1.1  only  partially  supports  this  hypothesis.  While  there 
was  a  significant  cost  for  the  HMD  for  detecting  the  unexpected  target  when  additional 
information  (cueing)  was  present  in  the  forward  field  of  view  (tanks),  there  was  no  difference 
between  the  two  displays  in  detection  accuracy  for  the  unexpected  target  in  a  “low”  cluttered 
scene  -  i.e.,  when  no  cueing  information  was  present  (soldiers). 

An  alternative  hypothesis,  more  consistent  with  the  full  data,  is  that  the  presentation  of 
cueing  information  may  have  directed  attention  to  a  certain  area  of  the  visual  scene,  hence 
precluding  scanning  of  the  surrounding  area.  If  this  were  the  case,  then  the  cost  of  detecting  the 
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unexpected  targets  using  the  HMD  relative  to  the  head  down  display  would  be  present  only  if 
cueing  were  present  also.  The  accuracy  data  presented  in  Figure  3.1.1  support  this  hypothesis. 
That  is,  the  augmented  reality  cueing  in  the  HMD  induces  a  sort  of  cognitive  tunneling  that  is 
reduced  when  cueing  is  presented  in  its  less  “real”  form  on  the  hand-held  display.  Additionally, 
two  other  factors  support  this  “tunneling”  hypothesis  of  cueing  effects:  the  limited  field  of  view 
provided  by  the  HMD  and  the  weight  of  the  HMD.  When  wearing  the  HMD,  subjects  could  only 
see  a  maximum  of  30°  of  the  visual  scene,  but  they  could  see  up  to  120°  (i.e.,  the  field  of  view  of 
a  person  with  normal  vision)  when  using  the  hand-held  display.  Thus,  information  in  the 
periphery,  which  provides  information  as  to  where  the  eyes  should  move  next,  was  available 
when  using  the  hand-held  display  but  not  when  wearing  the  HMD.  Consequently,  subjects  may 
have  relied  more  on  the  cueing  information  provided  by  the  HMD  than  that  provided  by  the 
hand-held  display  to  find  the  target,  and  were  so  focused  on  the  cue  that  they  missed  the 
unexpected  target  even  when  it  appeared  within  their  visual  field.  On  the  other  hand,  when  using 
the  hand-held  display,  subjects  were  only  directed  to  the  general  location  of  the  target  and  thus 
were  more  likely  to  scan  a  wider  area  around  the  cued  target  and  detect  the  unexpected  target. 
Note  that  the  difference  in  the  field  of  view  of  the  HMD  can  not  otherwise  account  for  the  misses 
for  detecting  the  nuclear  device,  because  the  nuclear  device  was  presented  within  15°  of  the 
target.  Furthermore,  if  this  were  the  case,  we  would  have  expected  a  reduction  in  nuclear  device 
detection  for  both  cued  and  uncued  HMD  trials.  Figure  3.1.1  suggests  that  the  cost  was  only 
present  for  cued  trials. 

The  second  factor  for  the  HMD  detection  accuracy  disadvantage  is  supported  by  the 
visual  scanning  data  shown  in  Figure  3.6.2.  The  weight  of  the  HMD  reduced  the  amount  of 
scanning  of  the  environment  and  may  have  encouraged  subjects  to  follow  the  cue  to  detect  a 
target  as  quickly  as  possible  (as  supported  by  the  detection  time  data  in  Figure  3.1.1),  whether  it 
was  the  high  priority  one  or  not. 

Thus,  the  data  suggests  that  the  hand-held  advantage  in  detecting  the  unexpected  objects 
may  be  due  to  a  cost  in  directing  attention  to  different  areas  of  the  visual  scene,  i.e.,  a  failure  to 
scan.  This  is  not  a  cost  that  has  been  extensively  addressed  by  other  HMD  studies,  although 
Seagull  and  Gopher  (1997)  did  observe  the  inhibiting  effects  on  scanning  and  attention  allocation 
imposed  by  HMDs  in  a  flight  simulation.  The  results  for  unexpected  event  detection  replicate  the 
findings  of  Yeh,  Wickens,  and  Seagull  (1998),  which  showed  that  the  unexpected  target  was 
detected  more  often  when  presented  in  conjunction  with  an  uncued  target  versus  a  cued  target, 
and  extend  the  findings  as  to  how  display  manipulations  can  reduce  cognitive  tunneling.  In  this 
study,  while  the  use  of  world-referenced  imagery  facilitated  detection  of  unexpected  events 
relative  to  the  use  of  screen-referenced  (or  non-conformal)  imagery,  the  current  results  show  that 
using  a  hand-held  display  facilitates  performance  relative  to  a  world-referenced  HMD. 

In  general,  targets  were  detected  faster  when  subjects  were  using  the  hand-held  display 
rather  than  the  HMD,  but  this  benefit  was  limited  to  those  instances  when  the  targets  were 
uncued  (Figure  3.2.1).  That  is,  for  cued  targets  (the  land  mines),  there  was  little  difference  in 
detection  times  due  to  display.  The  disadvantage  for  the  HMD  may  be  a  result  of:  (1)  the 
additional  clutter  in  the  forward  field  of  view  (e.g.,  the  secondary  task  and  occasionally,  the 
heading  information)  may  have  increased  the  difficulty  in  target  detection  (May-Ververs  & 
Wickens,  1998;  Teichner  &  Mochamuk,  1979),  or  (2)  the  limited  field  of  view  provided  by  the 
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HMD,  which  has  been  shown  to  lead  to  poorer  target  detection  performance  (Hettinger,  Nelson, 
&  Haas,  1994). 

If  the  first  factor  were  responsible  for  the  results,  we  might  expect  that  the  cost  of  clutter 
on  the  HMD  to  be  enhanced  by  targets  of  low  salience  (e.g.,  the  mines)  and  low  expectancy  (the 
nuclear  devices)  in  relation  to  those  of  higher  saliency  and  expectancy.  This  hypothesis  is  based 
on  the  findings  regarding  the  problems  of  low  salience  of  rare  event  detection  with  the 
superimposed  imagery,  characteristic  of  head-up  displays  (Martin-Emerson  &  Wickens,  1997; 
Wickens  &  Long,  1995). 

If,  on  the  other  hand,  the  second  factor  -  the  limited  field  of  view  -  were  responsible,  we 
would  expect  a  greater  HMD  time  cost  for  the  detection  of  more  salient  (and  therefore  more 
available  to  peripheral  vision)  uncued  soldiers  than  for  the  less  salient  uncued  land  mines, 
relative  to  the  hand-held  display.  That  is,  the  low  salience  of  the  target  could  result  in  its  inability 
to  be  seen  in  peripheral  vision  when  using  the  hand-held  display.  The  current  data  are  actually 
silent  with  regard  to  which  of  these  hypotheses  might  be  true,  since  detection  times  for  the  HMD 
was  approximately  5  seconds  slower  for  the  hand-held  display  for  both  the  more  salient  soldier 
(Figure  3.1.1b)  and  the  less  salient  land  mine  (Figure  3.2.1b).  Thus,  it  is  possible  that  both 
factors  were  operating,  and  that  their  net  effects  offset  each  other. 

The  results  for  target  detection  revealed  no  main  effect  of  the  wall  on  which  the  target 
was  presented,  although  the  significant  target  type  by  wall  interaction  reveals  substantial  cost  to 
detecting  the  uncued  mine  on  the  left  well.  Yeh,  Wickens,  and  Seagull  (1998)  found  that  in 
searching  for  the  target,  subjects  moved  their  head  clockwise,  similar  to  the  pattern  used  in 
reading  text  from  left  to  right.  Subjects  may  have  turned  their  head  from  the  center  wall,  where 
the  head  was  positioned  at  the  start  of  each  trial,  to  the  right  immediately  after  the  trial  began.  If 
the  target  was  not  found  on  the  right  wall,  subjects  then  moved  their  head  back  to  the  center  and 
searched  that  wall  for  objects  before  examining  the  left  wall.  Due  to  the  low  salience  of  the 
uncued  mine,  it  was  relatively  difficult  to  detect;  consequently,  subjects  may  have  needed  to 
examine  the  right  and  center  walls  more  closely  (i.e.,  for  a  longer  amount  of  time)  before  turning 
their  heads  to  examine  the  left  wall,  thereby  increasing  the  detection  time.  On  the  other  hand, 
when  cueing  is  present,  habit  no  longer  directs  search;  rather,  the  subject  follows  the  cue  to  a 
specific  area  of  the  display  and  detects  the  target;  this  could  explain  the  absence  of  a  wall  effect 
for  the  cued  targets  (tanks  and  cued  mines). 

The  results  of  the  target  identification  task  (Figure  3.4.1)  reflect  the  negative  effects  of 
clutter  resulting  from  superimposing  information  on  the  forward  field  of  view  and  warn  of  the 
potential  to  obscure  critical  information.  As  Figure  3.4.1  shows,  the  presence  of  the  lock-on 
reticle  superimposed  over  the  target  imposed  a  cost  on  accuracy  in  identifying  the  target  (tank)  as 
friend  or  foe  in  the  head-up  condition  relative  to  the  head-down  condition.  Further  evidence  to 
support  this  hypothesis  is  that  this  difference  in  accuracy  between  HMD  and  HHD  was  not 
present  for  the  uncued  target  (the  soldier),  i.e.,  the  case  in  which  no  reticle  was  present  in  the 
HMD  condition.  Thus,  superimposing  the  reticle  over  the  tank  increased  the  difficulty  of 
determining  which  way  the  barrel  of  the  tank  was  pointing,  the  critical  feature  for  the 
identification  task. 
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In  contrast  to  the  benefit  of  hand-held  display  for  target  identification,  its  use  imposed  a 
noteworthy  cost  to  reporting  the  target’s  azimuth.  In  the  current  experiment,  although  some 
amount  of  visual  scanning  was  required  by  subjects  when  using  either  the  HMD  or  hand-held 
display,  the  cost  (distance)  of  head-down  scanning  to  the  hand-held  display  was  greater  than  the 
cost  of  upward  scanning  to  the  HMD  horizon  line  since  the  horizon  line  would  sometimes  be 
present  in  the  field  of  view  simultaneously  with  the  target.  Additionally,  since  the  hand-held 
display  was  closer  to  the  subject  than  the  far  domain,  visual  accommodation  was  necessary  to 
bring  the  heading  information  into  focus.  On  the  other  hand,  information  viewed  through  the 
HMD  was  presented  at  the  same  distance  as  objects  in  the  far  domain. 

Thus,  the  combined  results  from  the  detection  times  and  target  identification  and  heading 
tasks  show  both  a  cost  for  clutter  (which  predicts  the  HMD  cost  on  the  low  salience  uncued 
mines  and  the  identification  accuracy  disadvantage  for  the  cued  targets)  and  a  cost  for  restricted 
field  of  view  (which  predicts  the  greatest  cost  on  the  detection  of  the  more  peripherally  visible 
soldiers)  on  the  presentation  of  information  head-up.  For  the  HMD,  the  reduced  scanning  when 
integrating  information  between  the  near  and  far  domains  facilitated  subjects’  ability  to  provide 
heading  information  but  severely  hindered  the  detection  of  the  unexpected,  high  priority,  nuclear 
devices. 

The  findings  from  the  secondary  task  showed  no  difference  in  the  accuracy  of 
performance  due  to  display  but  a  difference  attributable  to  target  cueing,  such  that  subjects 
detected  events  on  the  frequency  jamming  task  more  often  when  the  target  they  were  searching 
for  was  not  cued  than  when  it  was.  Note  that  the  nature  of  these  results  are  similar  to  those 
discussed  earlier  for  the  detection  of  nuclear  devices;  that  is,  in  both  cases,  the  presence  of 
cueing  imposed  a  cost  on  performing  a  concurrent  task.  The  results  for  the  secondary  task 
suggest  that  attention  was  allocated  in  a  task  dependent  rather  than  a  display  dependent  fashion. 
Thus,  the  presentation  of  cueing  pulled  attention  allocated  to  search,  and  away  from  the 
secondary  task,  and  it  made  no  difference  whether  the  intensified  search  domain  was  near  the 
cueing  reticle  (as  with  the  HMD)  or  far  away  (as  with  the  hand-held  display).  In  either  case,  the 
added  emphasis  to  the  search  task,  induced  by  the  cueing  reticle,  appeared  to  divert  attention 
from  the  secondary  task. 

Finally,  the  results  for  the  global  post-task  recognition  task  showed  that  subjects’  mental 
representations  of  the  location  of  objects  in  the  environment  was  aided  when  they  used  the  HMD 
rather  than  the  hand-held  display.  The  results  of  Yeh,  Wickens,  and  Seagull  (1998)  revealed  no 
difference  in  performance  for  this  task  between  the  HMD  with  world-referenced  and  screen- 
referenced  imagery,  suggesting  that  one’s  mental  representation  is  formed  not  on  the  basis  of 
display  manipulations  of  symbology  but  may  be  dependent  on  how  the  environment  is  viewed. 

In  the  current  study,  accuracy  on  this  task  was  poor  with  both  displays;  subjects  were  correct 
50%  of  the  time  when  using  the  HMD  condition  and  30%  with  the  hand-held  display,  whereas 
chance  performance  was  25%.  The  results  reveal  the  problems  in  using  long  term  memory  to 
recall  what  subjects  may  have  considered  incidental  information,  as  found  by  Wickens,  Liang, 
Prevett,  and  Olmos  (1996). 

Our  results  are  thus  inconsistent  with  previous  research,  which  shows  that  a  reduced  field 
of  view  impairs  one’s  ability  to  form  a  coherent  representation  of  the  world,  since  context, 
necessary  for  accurate  recognition,  is  lost  (National  Research  Council,  1997).  One  tentative 
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hypothesis  which  we  set  forth  that  may  account  for  our  findings  is  based  on  the  idea  of 
information  overload,  in  the  sense  that  one  can  only  use  a  limited  number  of  cues  to  develop  an 
picture  of  the  system  or  world,  a  heuristic  commonly  applied  in  decision  making  (Wickens, 

1992).  In  the  global  positioning  task,  what  was  important  was  not  how  many  different  views  of 
the  terrain  one  possessed  but  rather  knowledge  of  target  location.  Since  subjects  achieved  the 
same  level  of  performance  for  target  detection  using  both  the  HMD  and  hand-held  displays,  it  is 
reasonable  to  conclude  that  all  subjects  managed  to  attend  to  the  areas  of  the  display  containing 
the  most  relevant  information  with  respect  to  the  global  positioning  task  (i.e.,  the  target’s 
position  in  the  environment).  Thus,  even  though  the  HMD  limited  the  amount  of  information 
available  and  the  weight  reduced  the  amount  of  scanning  of  the  environment,  subjects  still 
attended  to  the  “important”  areas  of  the  display.  It  is  possible  that  reducing  the  amount  of  total 
information  allowed  subjects  to  better  piece  together  the  relevant  information  and  thus  create  a 
more  accurate  mental  representation  of  the  world.  On  the  other  hand,  when  subjects  scanned  the 
environment  freely  as  with  the  hand-held  display,  they  acquired  too  much  information  about  the 
terrain  from  moving  their  heads  freely  back  and  forth  and  as  a  consequence,  were  unable  to 
effectively  process  all  the  pieces.  These  unexpected  findings  need  to  be  examined  further. 

5.  CONCLUSION 

The  current  results  replicate  earlier  findings  of  the  cost-benefit  trade-off  of  cueing  (Yeh, 
Wickens,  &  Seagull,  1998).  That  is,  a  clear  benefit  for  targets  which  were  cued,  but  an  important 
cost  to  missing  high  priority  uncued  targets  in  the  same  scene. 

Importantly,  these  results  also  reveal  that  the  cost-benefit  trade-off  can  be  modified  or 
modulated  by  the  platform  on  which  cueing  is  presented.  The  use  of  a  hand-held  display  slightly 
reduces  the  benefits  of  cueing  but  greatly  reduced  the  cost  of  cueing  to  the  detection  of 
concurrent  uncued  high  priority  targets.  This  large  benefit  to  the  hand-held  display  (reducing  the 
cost  of  unexpected  target  detection)  was  partially  offset  by  a  smaller  cost  in  reporting  the 
azimuth  of  targets,  a  cost  attributed  to  scanning,  and  in  recalling  target  locations.  Thus,  an 
absolute  assessment  of  the  “better”  of  the  two  display  formats  evaluated  here  cannot  be  made 
without  considering  the  weights  on  the  relative  importance  of  the  different  tasks,  either  supported 
or  disrupted  by  the  display  format. 
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